(1) Field of the Invention
The present invention relates to picture coding techniques and picture decoding techniques, and particularly to picture coding techniques using a variable bit rate method and a variable frame rate method.
(2) Description of the Related Art
A variable bit rate method is one of conventional methods in compliance with picture coding technique, such as the MPEG (Moving Picture Expert Group) standard, for controlling a bit rate. This is a control method that tolerates local fluctuations of the bit rate caused by complexity of a picture to be coded and keeps the number of bits generated within a specified time period (1 second, for instance) constant mainly in order to observe constraints on capacity of a buffer memory for storing coded data and time for video recording on a recording medium. To control the number of bits to be allocated according to a scene, the number of bits of a scene which is difficult to be coded is increased (bit rate becomes higher) and the number of bits of a scene which is easy to be coded is decreased (bit rate becomes lower). In other words, this method encodes pictures so as to reproduce high-quality pictures, while the number of bits generated within a specified time period is maintained at a constant rate (within limited capacity of a buffer memory and limited recording time). (Refer to Japanese Patent Application Laid-Open No. 2001-25015)
A picture coding apparatus using the existing variable bit rate method, as mentioned above, is explained referring to FIG. 1 illustrating a picture coding apparatus 100. The picture coding apparatus 100 is comprised of an orthogonal transforming unit 105, a quantization unit 106, a variable length coding unit 107, an inverse quantization unit 108, an inverse orthogonal transforming unit 109, a frame memory 102, a motion estimation unit 103, a motion compensation unit 104 and a bit rate control unit 110.
The orthogonal transforming unit 105 performs discrete cosine transform (DCT) of a received frame signal to be coded 101 (picture signal data) per macroblock unit and generates DCT coefficients and outputs them to the quantization unit 106. Regarding a frame of I (Intra coded) picture, DCT operation is performed in an inter picture coding mode. Regarding a frame of P (Predictive coded) picture, DCT operation is performed in a forward prediction coding mode based on a preceding I picture or P picture. Regarding a frame of B (Bi-directional) picture, DCT operation is performed in a bi-directional prediction coding mode based on a preceding or following I picture or P picture.
The quantization unit 106 quantizes DCT coefficients inputted from the orthogonal transforming unit 105 per macroblock by quantization step received from the bit rate control unit 110 (a quantization parameter may be used) and outputs them to the variable length coding unit 107 and the inverse quantization unit 108. The variable length coding unit 107 performs variable length coding and multiplexing of the quantized DCT coefficients inputted from the quantization unit 106 and outputs them to an output buffer (not shown in this figure).
The inverse quantization unit 108 performs inverse quantization operation of the quantized DCT coefficients received from the quantization unit 106 and outputs them to the inverse orthogonal transforming unit 109. The inverse orthogonal transforming unit 109 reproduces picture signal data by performing inverse orthogonal transforming operation based on the inverse orthogonal transformed DCT coefficients inputted from the inverse quantization unit 108 and outputs them to the frame memory 102.
The frame memory 102 adds decoded picture signal data of an I picture or a P picture to the motion compensation data generated by the motion compensation unit 104 and stores the resulting data. The motion estimation unit 103 estimates motion vectors from a reference picture stored in the frame memory 102 and outputs the data showing the motion vectors to the motion compensation unit 104.
The motion compensation unit 104 generates motion compensation data (reference picture data) based on reference picture stored in the frame memory 102 for coding P pictures or B pictures and the data indicating motion vectors inputted from the motion estimation unit 103. The bit rate control unit 110 receives the generated number of bits from the variable length coding unit 107, determines the quantization step based on the generated number of bits, and transmits the quantization step to the quantization unit 106.
An overall control unit 140 is a part, which controls the picture coding apparatus 100 entirely, such as a microcomputer equipped with ROM and RAM. The overall control unit 140 controls timing of each process based on signals such as control signals.
FIG. 2 is a block diagram showing functional configuration related to the bit rate control unit 110 of the conventional picture coding apparatus 100. As FIG. 2 shows, the bit rate control unit 110 is comprised of a frame group target bits number calculating unit 111, a next frame target bits number calculating unit 112 and a quantization step calculating unit 113.
The frame group target bits number calculating unit 111 receives the number of generated bits, Nn 131, from the variable length coding unit 107 and stores it in an internal memory (not shown in this figure). At this operation, the frame group target bits number calculating unit 111 counts the number of receiving the number of generated bits, Nn 131 (that is, the number of coded frames). In addition, the frame group target bits number calculating unit 111 calculates the number of allocatable bits in frame unit for frames which are not yet coded, and transmits the resulting number to the next frame target bits number calculating unit 112 and sequentially updates of the number of allocatable bits, mentioned above, using the number of bits, Nn 131, generated from actual coding. “The frame group” mentioned here means a group of frames coded within a specified time period.
The next frame target bits number calculating unit 112 calculates the target number of bits to be allocated to a next frame based on the number of bits allocatable in every frame group received from the frame group target bits number calculating unit 111, and transmits the resulting target number to the quantization step calculating unit 113. Above target value is calculated by dividing the number of bits allocatable in every frame group at a specific time by the number of residual frames.
The quantization step calculating unit 113 calculates a quantization step 141 (a quantization parameter can be used) based on the target number of bits to be allocated to a next frame received from the next frame target bits number calculating unit 112, and outputs the resulting value to the quantization unit 106.
FIG. 3 is a flowchart showing process flow related to the overall control unit 140 and the control unit 110 of the conventional picture coding apparatus 100.
First, when the number of generated bits, Nn 131, is received at the bit rate control unit 110 (S1401), the overall control unit 140 judges if a frame to be coded next is the first frame of a frame group (S1402). In this case, if a next frame is the first frame of the frame group (Yes in S1402), the bit rate control unit 110 initializes the number of allocatable bits, Na, a time period to be allocated, Ta, and the number of frames to be allocated, Nt (S1405˜1407). “NA” mentioned here is an initial value of the number of bits to allocate to every frame group. “TA” is a period of whole frame group. “Rf” is a frame rate for coding used by the picture coding apparatus 100.
On the other hand, if a next frame is not the first frame of the frame group (No in S1402), the bit rate control unit 110 updates the number of allocatable bits, Na, and a time period to be allocated, Ta (S1403˜1404).
Next, the bit rate control unit 110 calculates the number of bits to be allocated, Nb, based on the number of allocatable bits, Na, and a frame to be allocated, Nt (S1408), and decrements the number of a frames to be allocated, Nt (S1409).
After that, the bit rate control unit 110 judges if a frame to be coded needs to be thinned out, and if it is necessary, the bit rate control unit 110 informs the overall control unit 140 (Yes in S1410). On the other hand, if thinning is not necessary (No in S1410), a quantization step is calculated and outputted to the quantization unit 110 (S1411).
The overall control unit 140 and the bit rate control unit 110 repeat above process until coding process is completed (S1401˜1412).
FIG. 4 shows an actual example of a calculation method for calculating the number of bits to be allocated, Nb, in the conventional bit rate control unit 110. In this case, the number of generated bits, Nn 131, and the number of bits to be allocated, Nb, should match for convenience sake.
It should be assumed that a picture signal data inputted by 30 [fps] (a frame signal to be coded 101) is coded at the frame rate of 15 [fps] and a frame group comprises 15 frames is essentially generated per second in the example of FIG. 4.
In this example, the frame signal to be coded 101 to be inputted is indicated as “an input picture frame number 1701” and a coded frame signal 121 is indicated as “a frame group frame number 1702”. A symbol “x” in columns of the frame group frame number 1702 and the number of generated bits, Nn 131, in FIG. 4 shows that coding is not performed for a corresponding frame of the input picture frame number 1701 but “thinning” is performed.
The number of generated bits, Nn, corresponding to an input picture frame number “9” (that is, a frame group frame number “5”), for example, is ((3600−1180)/11=220 bits) since the total number of bits already allocated to four frames is 1180 bits and the number of residual frames are “11”.
On the other hand, since an input picture frame number “11” is thinned out and the total number of bits already allocated is 1400 bits and the number of residual frames is “9”, the number of generated bits corresponding to an input picture frame number “13” (that is, a frame group frame number “6”) is ((3600−1400)/9=244 bits [truncated after decimal point]).
For information, the number “(14)” in the column of a frame group frame number 1702 corresponding to an input picture frame number “29” in FIG. 4 is a value of the last frame group frame number 1702 predicted at the point when an input picture frame number 1701 is “13”.
As described above, the conventional picture coding apparatus 100 allocates a bit to a following frame to be coded.
However, successive (complex) frames which are difficult to be coded result in the greater number of bits generated for coding even if a value of a quantization step is changed to a larger value (in this case, a value of a bit rate becomes greater). As a result, the number of bits to be allocated to frames following successive frames which are difficult to be coded is decreased in order to keep the number of bits generated within a specified time period (a group of frame) constant. Therefore a quantization step for coding the following frames should be bigger and picture quality of those frames is extremely degraded. This is the first problem. (In this case, a bit rate becomes significantly lower.)
FIG. 5 is a diagram showing the first problem of the conventional technology. It is defined that fifteen frames between a frame 1601 and a frame 1602 form a frame group and 3600 bits per frame group are allocated as a target value (as shown in FIG. 4 mentioned above). In this case, frames which are difficult to be coded are continued from a time period t1, and when a great number of bits are allocated for coding those frames, the number of bits to be allocated to the last half of a frame group becomes less and it is necessary to make a quantization step rough (this results in lower bit rate), however, picture quality is extremely degraded if a quantization step is roughened.
A quantization step between t3 and t4 becomes much coarse in FIG. 5 and it can be assumed that picture quality is degraded by the rough quantization step.
Furthermore, when the number of bits to be allocated becomes increasingly fewer and the frame rate should be changed at some midpoint in a frame group (that is, frames are thinned out), it is impossible to assign a great number of bits although the frame rate is lowered since the number of bits to be allocated to a frame is determined based on an initial frame rate even if the frame rate is changed. And it is necessary to lower the frame rate additionally or to make a quantization step rough additionally. This is the second problem.
FIG. 6 shows the second problem of the above conventional technology. FIG. 6 shows an example that a frame rate is gradually lowered since a great number of bits are allocated for coding frames which are hard to be coded, and the frame interval is getting bigger, and the motion of a picture becomes awkward, and thus the quality of the picture is degraded.